CLASSICAL TEST THEORY vs. ITEM RESPONSE THEORY An evaluation of the theory test in the Swedish driving-license test

The Swedish driving-license test consists of a theory test and a practical road test. The aim of this paper is to evaluate which Item Response Theory (IRT) model among the one (1PL), two (2PL) and three (3PL) parameter logistic IRT models that is the most suitable to use when evaluating the theory test in the Swedish driving-license test. Further, to compare the chosen IRT model with the indices in Classical Test Theory (CTT). The theory test has 65 multiple-choice items and is criterionreferenced. The evaluation of the models were made by verifying the assumptions that IRT models rely on, examining the expected model features and evaluating how well the models predict actual test results. The overall conclusion from this evaluation is that 3PL model is preferable to use when evaluating the theory test. By comparing the indices from CTT and IRT it was concluded that both give valuable information and should be included in an analysis of the theory test in the Swedish driving-license test. INDEX INTRODUCTION 1 AIM 2 METHOD: SAMPLE 3 METHOD: CLASSICAL TEST THEORY 3 METHOD: ITEM RESPONSE THEORY 4 1. Verifying the assumptions of the model 5 A. Unidimensionality 5 B. Equal discrimination 5 C. Possibility of guessing the correct answer 5 2. Expected model features 6 3. Model predictions of actual test results 6 Estimation methods 7 RESULTS: CLASSICAL TEST THEORY 8 Modeling the items 10 RESULTS: ITEM RESPONSE THEORY 10 1. Verifying the assumptions of the model 10 2. Expected model features 12 Invariance of ability estimates 12 Invariance of item parameter estimates 13 3. Model predictions of actual test results 16 Goodness of fit 16 Normal distributed abilities 16 Test information functions and standard error 17 Rank of the test-takers 18 COMPARING CTT AND IRT 20 DISCUSSION 22 FURTHER RESEARCH 24 REFERENCES 25 APPENDIX